Agreement, the F-Measure, and Reliability in Information Retrieval
نویسندگان
چکیده
منابع مشابه
Agreement, the f-measure, and reliability in information retrieval.
Information retrieval studies that involve searching the Internet or marking phrases usually lack a well-defined number of negative cases. This prevents the use of traditional interrater reliability metrics like the kappa statistic to assess the quality of expert-generated gold standards. Such studies often quantify system performance as precision, recall, and F-measure, or as agreement. It can...
متن کاملAn F-Measure for Context-based Information Retrieval
Computationally expensive processes, such as deductive reasoners, can suffer performance issues when they operate over large-scale data sets. The optimal procedure would allow reasoners to only operate on that information that is relevant. Procedures that approach such an ideal are necessary to accomplish the goal of commonsense reasoning, which is to endow an agent with enough background knowl...
متن کاملAgreement and Information in the Reliability of Coding
Coefficients that assess the reliability of data making processes – coding text, transcribing interviews, or categorizing observations into analyzable terms – are mostly conceptualized in terms of the agreement a set of coders, observers, judges, or measuring instruments exhibit. When variation is low, reliability coefficients reveal their dependency on an often neglected phenomenon, the amount...
متن کاملThe socio - cognitive theory in information retrieval (IR)
Abstract Background and Aim: The socio-cognitive theory introduced in information science by Horland and Alberchtsen. The socio-cognitive view turns the traditional cognitive program upside down. The socio-cognitive theory emphasizes on different cultural and social structures of users. Hence, the aim of the article is to explain the role of socio - cognitive theory in information retrieval (I...
متن کاملA measure theoretic approach to information retrieval
The vector space model of information retrieval is one of the classical and widely applied retrieval models. Paradoxically, it has been characterised by a discrepancy between its formal framework and implementable form. The underlying concepts of the vector space model are mathematical terms: linear space, vector, and inner product. However, in the vector space model, the mathematical meaning o...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of the American Medical Informatics Association
سال: 2005
ISSN: 1067-5027,1527-974X
DOI: 10.1197/jamia.m1733